Imaging system for vehicle

ABSTRACT

An imaging system for a vehicle includes an imaging sensor ( 12 ) and a display device ( 15 ) for displaying images representative of image data captured by the imaging sensor. The imaging sensor ( 12 ) has a rearward field of view when mounted at the vehicle. The imaging system generates a plurality of graphic overlays ( 130   a,    130   b,    130   c ) on the displayed images to enhance the driver&#39;s cognitive awareness of an object rearward of the vehicle. The graphic overlays comprises a plurality of graphic overlay segments. The graphic overlay segments convey three dimensional information to a person viewing the displayed images and the graphic overlays. The imaging system may adjust at least one of a color, an intensity and a rate of flashing of at least one of the plurality of graphic overlay segments in response to an object being detected rearward of the vehicle and within a distance threshold.

FIELD OF THE INVENTION

The present invention relates generally to imaging or vision systems,particularly for vehicles which provide a vehicle operator with wideangle scenic information of an area immediately around the vehicle, suchas rearward of the vehicle.

BACKGROUND OF THE INVENTION

A long felt need in the art of vehicle vision systems, such as rear-viewsystems, has been to provide an operator of a vehicle with wide anglescenic information of an area around the vehicle, such as directlyrearward of the vehicle, when the vehicle is traveling in reverse.Neither interior rear-view mirrors nor side exterior mirrors allow forvisibility of the area immediately rearward of the vehicle's bumper,which is the information most critical to the vehicle operator whenbacking up. Therefore, various camera-based rear view vision systemshave been proposed to provide visibility of this blind spot.

It is also known to provide a graphic overlay on the displayed image ofthe rearward scene to enhance the driver's perception of the rearwardfield of view. Examples of such graphic overlays are described in U.S.Pat. Nos. 5,670,935; 5,949,331; 6,222,447; and 6,611,202, which arehereby incorporated herein by reference in their entireties.

Various camera-based rear vision systems for vehicles backing up havebeen proposed. In one form of these systems, a camera with aconventional, standard lens is located on the rearward portion of thevehicle to provide a view of the area behind the vehicle. However,standard lenses fail to capture a wide angle view of the area, thusfailing to provide the vehicle operator with an image of the entirecritical area directly rearward of the vehicle. In order to provide awider angle view of the rearward area, a wide angle lens system may beused with the camera to capture the critical area. Examples of suchsystems are described in U.S. Pat. Nos. 6,922,292; 6,757,109; 6,717,610;6,590,719; and 6,201,642, which are hereby incorporated herein byreference in their entireties.

SUMMARY OF THE INVENTION

The present invention is directed toward enhancing exterior visibilityto the driver of a vehicle by providing an imaging system that providesa wide angle field of view of the area directly exteriorly of a vehicle,and that provides a graphic overlay superimposed on a displayed image toenhance the driver's viewing and understanding or cognitive awareness ofthe displayed image of the scene rearward of the vehicle and any objector objects detected therein. The present invention is directed towardsenhancing the interpretation of visual information in a rearview visionsystem by presenting information in a manner which does not requiresignificant concentration of the driver or present distractions to thedriver.

The rearview vision system of the present invention has an image capturedevice or image sensor or camera positioned on the vehicle and directedrearwardly with respect to the direction of travel of the vehicle, and adisplay for displaying images captured by the image capture device.Image enhancement means may be provided for enhancing the displayedimage, such as in the form of graphic overlays superimposed on thedisplayed image. Such graphic overlays may include indicia of theanticipated path of travel of the vehicle which is useful in assistingthe driver in guiding the vehicle in reverse directions, includingtrailering maneuvers. The graphic overlays may be flashed orintermittently activated and/or displayed in different colors dependingon a distance to a detected object rearward of the vehicle so as toprovide a spectral cue and a spatial cue to the driver as the drivermaneuvers the vehicle in a rearward direction. The graphic overlays maycomprise graphic overlay segments extending upward and inward along thesides of the image so as to be indicative of segments extending rearwardfrom the sides of the vehicle. The graphic overlay segments may conveythree dimensional information to a person viewing the displayed imagesand the graphic overlays. The graphic overlay segments may be viewed bythe driver as having a virtual height dimension to enhance the driver'scognitive awareness of the graphic overlays. The graphic overlaysegments may be displayed as three dimensional graphic overlay segmentshaving a virtual height and a virtual thickness.

Optionally, for rearward facing cameras, the image may be displayed as amirrored image or reverse image on the display so that the driver of thevehicle, when looking forward toward the image display, views thedisplayed image as if it is a reflected image in a rear view mirror, inorder to assist the driver with guiding the vehicle during a reversingmaneuver without having to turn his or her head around to look rearward.Optionally, the image processor or the display may process or adjust theimage data to achieve the desired reverse image display. Optionally, anyperspective and optical distortion in the captured images may beaccounted for or generally matched in the generated graphic overlays soas to provide enhanced or more accurate spatial cues to the driver.Optionally, the graphic overlays may be adjusted in response to anobject detection system so as to not display the graphic overlays atareas in the displayed image that are representative of a detectedobject.

The camera or image sensor of the imaging system may include or utilizea wide angle lens that is compact, durable and inexpensive tomanufacture. The image sensor and lens assembly of the present inventionthus may comprise a wide angle lens or lens assembly, such as amulti-element lens assembly having multiple optics to provide a wideangle field of view with reduced distortion. Optionally, the lens mayinclude multiple optics, such as four to seven optic elements, with anouter element (the element at the rearwardmost end of the lens andexposed at the rear exterior of the vehicle) comprising a glass elementand with at least one other element comprising a plastic asphericelement.

These and other objects, advantages, purposes, and features of thisinvention will become apparent upon review of the followingspecification in conjunction with drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of an imaging device comprising an imagesensor and lens in accordance with the present invention;

FIG. 2 is an image as displayed to a driver of the vehicle in accordancewith the present invention;

FIG. 3 is another image as displayed to a driver of the vehicle andhaving a graphic overlay of the vehicle's projected rearward trajectorysuperimposed thereon in accordance with the present invention;

FIG. 4 is another image as displayed to a driver of the vehicle andhaving a graphic overlay of the vehicle's projected rearward trajectorysuperimposed thereon and including relative distance markers inaccordance with the present invention;

FIG. 4A is a side perspective view of a vehicle and virtual overlaysegments of the present invention as they would appear if actuallypresent rearward of the vehicle;

FIG. 5 is another image as displayed to a driver of a vehicle and havinga trailer hitch alignment graphic overlay of the vehicle's projectedrearward trajectory superimposed thereon and including relative distancemarkers in accordance with the present invention;

FIG. 5A is a side perspective view of the graphic overlays of FIG. 5 asthey would appear if actually present rearward of the vehicle;

FIG. 6 is another image as displayed to a driver of a vehicle and havinga trailer hitch alignment graphic overlay of the vehicle's projectedrearward trajectory superimposed thereon similar to FIG. 5, with thetrailer hitch alignment overlay shown as a solid virtual object;

FIG. 6A is a side perspective view of the graphic overlays of FIG. 6 asthey would appear if actually present rearward of the vehicle;

FIG. 7 is another image as displayed to a driver of a vehicle and havinga trailer hitch alignment graphic overlay of the vehicle's projectedrearward trajectory superimposed thereon, with the trailer hitchalignment overlay shown as a two dimensional overlay;

FIG. 8 is another image as displayed to a driver of a vehicle and havinga trailer hitch alignment graphic overlay of the vehicle's projectedrearward trajectory superimposed thereon similar to FIG. 7, with thetrailer hitch alignment overlay shown as a solid two dimensionaloverlay;

FIG. 9 is another image as displayed to a driver of a vehicle and havinga trailer hitch alignment graphic overlay of the vehicle's projectedrearward trajectory superimposed thereon similar to FIG. 6, with thetrailer hitch alignment overlay shown as a virtual solid wedge-shapedobject;

FIG. 10 is another image as displayed to a driver of a vehicle andhaving a trailer hitch alignment graphic overlay of the vehicle'sprojected rearward trajectory superimposed thereon, with the relativedistance markers and the trailer hitch alignment overlay shown asvirtual solid objects;

FIG. 10A is a side perspective view of the graphic overlays of FIG. 10as they would appear if actually present rearward of the vehicle;

FIG. 11 is another image as displayed to a driver of a vehicle andhaving a trailer hitch alignment graphic overlay of the vehicle'sprojected rearward trajectory superimposed thereon, with the relativedistance markers and the trailer hitch alignment overlay shown asvirtual semi-transparent objects;

FIG. 11A is a side perspective view of the graphic overlays of FIG. 11as they would appear if actually present rearward of the vehicle;

FIG. 12 is another image as displayed to a driver of a vehicle andhaving a trailer hitch alignment graphic overlay of the vehicle'sprojected rearward trajectory superimposed thereon, with the relativedistance markers and the trailer hitch alignment overlay shown asvirtual semi-transparent objects, and with a lower surface of thevirtual distance markers being shown as a solid surface;

FIG. 13 is another image as displayed to a driver of a vehicle andhaving a trailer hitch alignment graphic overlay of the vehicle'sprojected rearward trajectory superimposed thereon similar to FIG. 12,with the lower and upper surfaces of the virtual distance markers beingshown as solid surfaces; and

FIG. 14 is another image as displayed to a driver of a vehicle andhaving a trailer hitch alignment graphic overlay of the vehicle'sprojected rearward trajectory superimposed thereon, with the relativedistance markers shown as vertically spaced apart two dimensionalsurfaces.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring now to the drawings and the illustrative embodiments depictedtherein, a wide angle imaging system provides an operator of a vehiclewith scenic information of an area immediately exteriorly, such asrearwardly, of the vehicle, which may be an automobile, a light truck, avan, a large truck, a sport utility vehicle or the like. The imagingsystem includes an image capture device 12 (FIG. 1) having an imagingsensor and a lens or lens assembly 14 that functions to focus a rearwardfield of view at an imaging plane of the imaging sensor or camera.Images 16 (FIG. 2) are displayed on a display device or screen withreduced distortion to enhance the driver's viewing and understanding ofthe displayed images. The rearview imaging system includes an imageprocessor for receiving data signals from the image capture device andsuperimposing the graphic overlay on the displayed image, as discussedbelow.

The imaging sensor for the vehicle vision system of the presentinvention may comprise any suitable sensor, and may utilize variousimaging sensors or imaging array sensors or cameras or the like, such asa CMOS imaging array sensor, a CCD sensor or other sensors or the like,such as the types described in U.S. Pat. Nos. 5,550,677; 5,670,935;5,760,962; 5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610;6,590,719; 6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176;6,559,435; 6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978;7,038,577; and 7,004,606; and/or U.S. patent application Ser. No.11/315,675, filed Dec. 22, 2005 and published Aug. 17, 2006 as U.S.Patent Publication No. US-2006-0184297A1; and/or Ser. No. 10/534,632,filed May 11, 2005 and published Aug. 3, 2006 as U.S. Patent PublicationNo. US-2006-0171704A1; and/or U.S. provisional applications, Ser. No.60/845,381, filed Sep. 18, 2007; and Ser. No. 60/837,408, filed Aug. 11,2006, and/or PCT Application No. PCT/US2003/036177 filed Nov. 14, 2003,and published Jun. 3, 2004 as PCT Publication No. WO 2004/047421, whichare all hereby incorporated herein by reference in their entireties. Thecontrol may include a lens element or optic between the image sensor andthe forward scene to substantially focus the scene at an image plane ofthe image sensor. Optionally, the optic may comprise a wide angle lensthat provides a proportionally distributed central portion of the imagedscene while providing classical optical distortion on the periphery ofthe imaged scene. Optionally, the optic may comprise anon-flat fieldcurvature, which may focus a generally central portion of the imagedscene onto the image sensor, while providing reduced but acceptablefocus on the periphery of the imaged scene. The imaging device maycomprise an image sensing module or the like, and may utilize aspectsdescribed in U.S. patent application Ser. No. 10/534,632, filed May 11,2005; and/or PCT Application No. PCT/US2006/041709, filed Oct. 27, 2006and published May 10, 2007 as International Publication No. WO07/053,404; and/or PCT Application No. PCT/US2003/036177 filed Nov. 14,2003 and published Jun. 3, 2004 as PCT Publication No. WO 2004/047421,which are hereby incorporated herein by reference in their entireties.

Such imaging sensors or cameras are pixelated imaging array sensorshaving a photosensing array of a plurality of photon accumulating orphotosensing light sensors or pixels, which are arranged in atwo-dimensional array of rows and columns on a semiconductor substrate.The camera established on the substrate or circuit board includescircuitry that is operable to individually access each photosensor pixelor element of the array of photosensor pixels and to provide an outputor image data set associated with the individual signals to the control,such as via an analog to digital converter. As the camera receives lightfrom objects and/or light sources in the target scene, the control maythen be operable to process the signal from at least some of the pixelsto analyze the image data of the captured image, as discussed below.

Optionally, the imaging sensor may be suitable for use in connectionwith other vehicle imaging systems, such as, for example, an objectdetection system or blind spot detection system, where a blind spotindicator may be operable to provide an indication to the driver of thehost vehicle that an object or other vehicle has been detected in thelane or area adjacent to the side of the host vehicle. In such a blindspot detector/indicator system, the blind spot detection system mayinclude an imaging sensor or sensors, or ultrasonic sensor or sensors,or sonar sensor, or radar, or LIDAR or sensors or the like. For example,the detection system may utilize aspects of the detection and/or imagingsystems described in U.S. Pat. Nos. 7,038,577; 6,882,287; 6,198,409;5,929,786; and/or 5,786,772, and/or U.S. patent application Ser. No.11/315,675, filed Dec. 22, 2005; and/or Ser. No. 11/239,980, filed Sep.30, 2005, and/or U.S. provisional applications, Ser. No. 60/696,953,filed Jul. 6, 2006; Ser. No. 60/628,709, filed Nov. 17, 2004; Ser. No.60/614,644, filed Sep. 30, 2004; and/or Ser. No. 60/618,686, filed Oct.14, 2004, and/or of reverse or backup aid systems, such as therearwardly directed vehicle vision systems described in U.S. Pat. Nos.5,550,677; 5,760,962; 5,670,935; 6,922,292; 6,590,719; 6,201,642;6,396,397; 6,498,620; 6,717,610; 6,757,109; and/or 7,005,974, and/or ofthe rain sensors described in U.S. Pat. Nos. 6,250,148 and 6,341,523,and/or of other imaging systems, such as the types described in U.S.Pat. Nos. 7,123,168; 6,353,392; and/or 6,313,454, with all of the abovereferenced U.S. patents, patent applications and provisionalapplications and PCT applications being commonly assigned and beinghereby incorporated herein by reference in their entireties.

In order to enhance the driver's understanding of what is occurring inthe area surrounding the vehicle, the rearview vision system includes adisplay device or display element having image enhancements (FIG. 3).The display device may comprise any suitable display device or element,such as a video display screen, such as a video display screen utilizingaspects of the display devices described in U.S. Pat. Nos. 5,530,240;6,329,925; 7,370,983; 7,274,501; 7,255,451; 7,195,381; 7,184,190;6,690,268; 5,668,663 and/or 5,724,187, and/or in U.S. patent applicationSer. No. 10/538,724, filed Jun. 13, 2005; Ser. No. 11/226,628, filedSep. 14, 2005; Ser. No. 10/993,302, filed Nov. 19, 2004; and/or Ser. No.11/520,193, filed Sep. 13, 2006; and/or PCT Application No.PCT/US03/29776, filed Sep. 9, 2003; and/or PCT Application No.PCT/US03/35381, filed Nov. 5, 2003; and/or PCT Application No.PCT/US2006/018567, filed May 15, 2006; and/or PCT Application No.PCT/US2006/042718, filed Oct. 31, 2005, and published May 10, 2007 asInternational PCT Publication No. WO 2007/053710A1; and/or PCTApplication No. PCT/US03/40611, filed Dec. 19, 2003, which are allhereby incorporated herein by reference in their entireties. Optionally,the display device or video display screen may be located at or in ornear the interior rearview mirror assembly of the vehicle. Optionally,the display screen may be located within the mirror assembly and behindthe mirror reflective element so that the display is transmitted throughthe reflective element for viewing by an occupant of the vehicle, or thedisplay screen may be covertly located behind the reflective element toprovide a DOD (Display-on-Demand) display through the transflectivereflective element, such that the presence of the video screen is onlynoticeable or discernible when the video screen is activated, such as byutilizing aspects described in U.S. Pat. Nos. 7,370,983; 7,274,501;7,184,190; 6,690,268; 5,668,663 and/or 5,724,187, and/or in U.S. patentapplication Ser. No. 10/538,724, filed Jun. 13, 2005; and/or Ser. No.11/226,628, filed Sep. 14, 2005, which are all hereby incorporatedherein by reference in their entireties.

Optionally, for such rearward facing camera applications, the capturedimages may be displayed as a mirrored image or reverse image on thedisplay so that the driver of the vehicle, when looking forward towardthe image display, views the displayed image as if it is a reflectedimage in a rearview mirror, in order to assist the driver with guidingthe vehicle during a reversing maneuver without having to turn his orher head around to look rearward. Optionally, the image processor or thedisplay may process or adjust the image data to achieve the desiredreverse image display, such as by utilizing aspects of the systemsdescribed in U.S. Pat. Nos. 5,550,677; 5,670,935; 5,949,331; 6,222,447;5,760,962; 5,715,093; 5,877,897; 6,922,292; 6,757,109; 6,717,610;6,590,719; 6,201,642; 6,498,620; 5,796,094; 6,097,023; 6,320,176;6,559,435; 6,831,261; 6,806,452; 6,396,397; 6,822,563; 6,946,978;7,038,577; and/or 7,004,606, which are hereby incorporated herein byreference in their entireties.

In the illustrated embodiment, the image enhancements of the imagingsystem and/or display device include graphic overlays 20 a, 20 b whichare lines or segments intended to illustrate to the driver theanticipated path of rearward movement of the vehicle based on thepresent trajectory of the vehicle. In the illustrated embodiment of FIG.3, the graphic overlays 20 a, 20 b indicate the anticipated vehiclemotion as a function of the vehicle direction of travel as well as therate of turn of the vehicle. Optionally, the displayed image may have atextual message 20 c alerting the driver of a detected object orotherwise instructing the driver during the rearward maneuver.

The forward or rearward direction of vehicle travel is determined inresponse to the operator placing the gear selection device (not shown)in the reverse gear position. The degree of turn of the vehicle may bedetermined by monitoring the movement of the vehicle steering system,monitoring the output of an electronic compass, or monitoring thevehicle differential drive system or the like. Optionally, anddesirably, if the vehicle is not in reverse gear position, the graphicoverlays are not presented. The imaging system and graphic overlays mayutilize aspects of the systems described in U.S. Pat. Nos. 5,670,935;5,949,331; 6,222,447; and 6,611,202, which are hereby incorporatedherein by reference in their entireties. Optionally, the graphicoverlays may be static overlays (in other words, graphic overlays thatextend generally rearwardly toward a focal point of expansion rearwardof the vehicle and that are not dependent on a steering wheel angle orthe like), while remaining within the spirit and scope of the presentinvention.

Optionally, and as shown in FIGS. 4 and 4A, the graphic overlays 30 a,30 b may function to enhance the driver's awareness of an objectdetected rearward of the vehicle when the vehicle is shifted into areverse gear. Optionally, the graphic overlays may include a centrallylocated overlay 30 c, such as a dashed line or solid or opaque orsemi-transparent or transparent surface or the like, that extends upwardtoward the focal point of expansion of the displayed image. In theillustrated embodiment, each graphic overlay 30 a, 30 b may be providedin three different colors, such as a green segment 32, a yellow segment34 and a red segment 36, with the upper or further out segment 32 beinggreen (or other suitable color or pattern), the middle segment 34 beingyellow (or other suitable color or pattern), and the closest or lowestsegment 36 being red (or other suitable color or pattern). Theindividual segments may be activated or overlaid in response to adetection of an object rearward of the vehicle and a detected ordetermined distance between the rear of the vehicle and the detectedobject.

For example, the rearward imaging system of the present invention mayinclude an imaging sensor for capturing images in a rearward field ofview and a display for displaying the images. The system may include ormay operate in conjunction with an object detection system for detectingobjects rearward of the vehicle and in the path of the vehicle when thevehicle is traveling in reverse (such as, for example, an ultrasonicsensing system, a lidar sensing system or a radar sensing system or viaimage processing of the captured images or the like). When the vehicleis reversing and no object is detected by the object detection system(or if an object is detected that is further away from the vehicle thana predetermined threshold distance), the graphic overlay or overlays maynot be displayed or may be displayed as non-colored or dark or neutrallines or continuous segment or the like (such as similar to the graphicoverlays 20 a, 20 b shown in FIG. 3). As the vehicle approaches theobject so that the object is detected by the object detection system (orwhen a detected object is within a first threshold distance from therear of the vehicle but outside of a second threshold distance from therear of the vehicle), the outer graphic overlay segment 32 may flash ormay be displayed as green to alert the driver of the presence of theobject, while also notifying the driver that the object is still a safedistance rearward of the vehicle. As the vehicle further approaches theobject (for example, when the object is within the second thresholddistance from the rear of the vehicle but outside of a third thresholddistance from the rear of the vehicle), the middle graphic overlaysegment 34 may flash or may be displayed as yellow to alert the driverthat the vehicle is approaching the object. As the vehicle furtherapproaches the object (for example, when the object is within the thirdthreshold distance from the rear of the vehicle), the inner graphicoverlay segment 36 may flash or may be displayed as red to alert thedriver that the vehicle is very near to the object.

Optionally, the system may also provide a tone or audible alert when thevehicle and object are within the third threshold separation distance orwhen the vehicle and object are less than a fourth threshold distanceapart to further alert the driver that the object is very near to thevehicle and that the driver should not back up any further. Optionally,the graphic overlays may also or otherwise vary in shape or pattern orthickness or color to enhance the driver's cognitive awareness of thelocation of and distance to a detected object rearward of the vehicle.Optionally, as one segment is activated or colored (as the object isdetected within the respective range from the rear of the vehicle), theother segment or segments may be deactivated or de-colored, so that thedriver can readily discern that the object is detected within theparticular range encompassed by the currently activated or coloredgraphic overlay segments.

Optionally, the graphic overlay may be designed or adjusted or selectedto assist colorblind drivers (or others that have different perceptionof colors than the general population) to distinguish the color segmentsof the overlay. A majority of the colorblind population are “red-greencolorblind” and have problems in either in the red or green opsin gene.For example, people with a mutant red opsin gene are called“protanopes”, while people with a mutant green opsin gene are called“deuteranopes”. It is estimated that for males, about 8 percent ofCaucasians, about 5 percent of Asians and about 4 percent of Africansare “red-green” colorblind. Color blindness is typically sex-linked, soa reduced number of females are colorblind. Red-green color blind peoplehave difficulty distinguishing colors between red and green with similarintensity or brightness. To address this problem, for example, thegraphic overlays of the present invention may replace the red coloredsegment or segments in the overlay graphics with a reddish purple orvermilion colored segment or segments, and may replace the green coloredsegment or segments with a blue or bluish green segment or segments.Such colors may be recognized and discerned by a typical red-greencolorblind person (the yellow colored segments may remain yellow sincetypical red-green colorblind people would be able to recognize anddiscern the yellow colored segments). Optionally, different texturalfeatures or patterns (such as different stippling or cross-hatching ordifferent degrees of transparency or translucency of the segments or thelike) can be used to assist colorblind drivers to distinguish differentsegments of the graphic overlay.

Optionally, the different colors or features or patterns of the segmentsmay be selectively provided in response to the user or driver selectingthe desired or appropriate graphic overlay display scheme. For example,a software or hardware toggle switch or other suitable user actuatableinput (such as a button or a voice recognition system or the like) maybe implemented to allow the user or driver of the vehicle to select asuitable overlay color scheme, such as a display displaying normalcolored graphic overlays for non-colorblind drivers or a displaydisplaying modified graphic overlays (such as a different color schemeor a different texture or pattern scheme or the like) for colorblinddrivers. The switch may offer different choices for the user, such as acolor scheme for a red-green colorblind person or a different colorscheme for other types of colorblindness (whereby the system adjusts thecolor scheme of the graphic overlays accordingly).

Optionally, and with reference to FIGS. 5-14, the present inventionprovides a graphic overlay, such as a generally centrally locatedalignment overlay, to assist the driver of the vehicle to back up to atrailer hitch or the like. Optionally, and desirably, and as can be seenwith reference to FIG. 4A, the camera or image sensor 12 at the rear ofthe subject vehicle 13 may be offset from the centerline 13 a of thevehicle 13 and angled so as to have its field of view be generallyrearward and toward the centerline of the vehicle. Such a cameraorientation conveys three dimensional information to the person viewingthe display by allowing for a three-dimensional or perspective image ofa trailer hitch alignment graphic overlay that may be imaged ordisplayed on a display device 15 (such as a video display screen or thelike, and such as a display screen located at or near the interiorrearview mirror of the vehicle) so as to appear to extend rearward fromthe rear of the vehicle and generally along the centerline of thevehicle. This arrangement allows the diver of a vehicle to identify boththe vehicle path center on or along the ground and the trailer hitchpath above the ground when a camera or image sensor is offset from thecenterline of the subject vehicle. Optionally, the center alignmentgraphic overlay may be displayed so that a person viewing the displayedimage perceives the alignment graphic overlay to be at a differentheight from the sideward relative distance graphic overlays, in order toconvey three dimensional information to the person viewing the display.

For example, and as shown in FIGS. 5 and 5A, a graphic overlay includesa pair of sideward overlays 130 a, 130 b and a generally centrallylocated trailer hitch alignment overlay 130 c. The side overlays 130 a,130 b may be substantially similar to the side overlays 30 a, 30 b,discussed above, and may have colored segments and the like, such that adetailed discussion of the side overlays need not be included herein.The hitch alignment overlay 130 c is generally at or near the center ofthe displayed image and extends upward toward the focal point ofexpansion of the displayed image to indicate the predicted path of atrailer hitch. The side overlays 130 a, 130 b may function as describedabove, and may be activated or generated or displayed when the vehicleis shifted into reverse, while the hitch alignment overlay 130 c may besimilarly activated or generated or displayed, or may be activated orgenerated or displayed in response to a user input, such as a controlinput or button or the like that is activated by the driver of thevehicle when the driver is about to back up the vehicle to a trailer (ormay be automatically activated or generated or displayed in response tothe system detecting or identifying a trailer rearward of the vehiclewhen the vehicle is traveling in reverse), while remaining within thespirit and scope of the present invention.

As can be seen in FIGS. 5 and 5A, the displayed image is captured by animage sensor that is offset from the centerline of the vehicle and isdirected rearward and toward the centerline of the vehicle. In theillustrated embodiment, image sensor is disposed about 305 mm offsetfrom the centerline of the vehicle at about 875 mm above the ground andoriented at an angle of about 40 degrees toward the centerline of thevehicle. The hitch alignment overlay 130 c is displayed as athree-dimensional shape that extends upward in the image and generallyalong the centerline of the vehicle and toward the focal point ofexpansion of the camera. The hitch alignment overlay 130 c is generatedso as to appear to be a three dimensional object at the rear of thevehicle (such as an object or alignment element having a height of about440 mm or thereabouts to approximate a height of a hitch of thevehicle). Thus, the alignment overlay provides two planes, one generallyat the ground level and one generally at a typical trailer hitch height.

Other mounting locations and angles for the camera and/or otherdimensions of the graphic overlays may be implemented without affectingthe scope of the present invention. Because the camera is preferablydisposed at a height greater than the virtual height of the alignmentelement/overlay, the displayed image provides a perspective viewrearward and downward toward the three dimensional alignment overlay sothat the driver can readily discern and understand the alignment overlayto assist the driver in reversing the vehicle toward a trailer hitch ofa trailer or the like. The alignment overlay and the offset camera thusprovide a scaled and distorted three dimensional image to provide thedriver improved depth perception for driving the vehicle toward thetrailer or other targeted object.

In the illustrated embodiment of FIGS. 5 and 5A, alignment overlay 130 cis a wire frame or line drawing or representation of a perspective viewof a three dimensional rectangular form or virtual three dimensionalobject or overlay. However, other shapes or forms may be utilized toprovide the desired perspective or view to the driver to assist thedriver in reversing the vehicle toward a trailer hitch or other object.For example, and as shown in FIGS. 6 and 6A, the side overlays 130 a′,130 b′ may be similar to side overlays 130 a, 130 b, discussed above,while the alignment overlay 130 c′ may be a solid or colored threedimensional rectangular form or virtual three dimensional object oroverlay to further enhance the viewability and discernability of thealignment overlay to the driver of the vehicle. Optionally, for example,and as shown in FIG. 7, the side overlays 230 a, 230 b may be similar toside overlays 130 a, 130 b, discussed above, while the alignment overlay230 c may be a generally planar, two dimensional rectangular form (suchas only the upper portion or surface of the three dimensionalrectangular form of alignment overlay 130 c, discussed above), which maybe a wire form representation (as shown in FIG. 7), or the alignmentoverlay 230 c′ may be a solid or colored representation (as shown inFIG. 8), depending on the particular application and desired appearanceof the alignment overlay in the displayed image. In the illustratedembodiments of FIGS. 7 and 8, the two dimensional alignment overlay 230c is displayed so that, when a person is viewing the displayed image andgraphic overlays, the person perceives the alignment overlay 230 c to beat a different height (such as higher) than the two dimensional sideoverlays 230 a, 230 b (which appear to extend along the ground or roadsurface rearward of the vehicle) to convey three dimensional informationto the person viewing the display. Optionally, and as shown in FIG. 9,the side overlays 330 a, 330 b may be similar to side overlays 130 a,130 b, discussed above, while the alignment overlay 330 c may be a prismor wedge shaped form having an upper surface (such as a surface orportion similar to the upper region of the alignment overlay 130 c,discussed above) and a shape that tapers or narrows downward togenerally a pointed or narrowed lowered portion, to further enhance theperspective view of the alignment overlay. The alignment overlay 330 cmay be colored or shaded or solid (as shown in FIG. 9), or may be a wireform representation or the like, without affecting the scope of thepresent invention.

Optionally, the side overlays may also be represented or generated ordisplayed three dimensionally to further assist the driver of thevehicle during the rearward maneuver. For example, and as shown in FIGS.10 and 10A, the side overlays 430 a, 430 b may appear as threedimensional colored or shaded or solid “walls” extending rearward fromthe vehicle to assist the driver in guiding or driving the vehicletoward the trailer or targeted object. The side overlays 430 a, 430 bmay be otherwise similar to side overlays 30 a, 30 b, and may havecolored segments 432, 434, 436 or the like, such as discussed above(optionally, the alignment overlay 430 c may be similarly colored orpatterned if desired). Optionally, and as shown in FIGS. 11 and 11A, theside overlays 430 a′, 430 b′ and alignment overlay 430 c′ may bedisplayed as wire frame representations and/or as partially transparentcolored representations or virtual three dimensional objects or overlaysso as to provide the desired color and enhanced viewability anddiscernability while limiting or reducing any potential interferencewith the driver's view of other objects in the displayed image.Optionally, and as shown in FIG. 12, the side overlays 430 a″, 430 b″and alignment overlay 430 c″ may be displayed as wire framerepresentations and/or as partially transparent colored representations,while the side overlays 430 a″, 430 b″ may include a lower portion thatis substantially similar to side overlays 30 a, 30 b to further enhancethe viewability and discernability of the side overlays while limitingor reducing any potential interference with the driver's view of otherobjects in the displayed image. Optionally, and as shown in FIG. 13, theside overlays 430 a′″, 430 b′″ and alignment overlay 430 c″ may bedisplayed as wire frame representations and/or as partially transparentcolored representations, while the side overlays 430 a′″, 430 b′″ mayinclude a lower portion that is substantially similar to side overlays30 a, 30 b and an upper portion that is also colored or shaded togenerally correspond with the lower portion of the side overlays, inorder to provide enhanced viewability and discernability of the sideoverlays while further limiting or reducing any potential interferencewith the driver's view of other objects in the displayed image.Optionally, and as shown in FIG. 14, the side overlays 530 a, 530 b mayhave an upper portion 531 a and a lower portion 531 b similar to sideoverlays 430′″, 430′″, but may not have the wire form or shaded regiontherebetween. The upper and/or lower portions 531 a, 531 b of sideoverlays 530 a, 530 b may have colored segments to assist in indicatinga distance to the targeted object. Optionally, and as also shown in FIG.14, the hitch alignment overlay 530 c may comprise a two dimensionalrectangular form (either shaded or colored or wire faith), such asdescribed above. In the illustrated embodiment of FIG. 14, the alignmentoverlay 530 c is displayed so that, when a person is viewing thedisplayed image and graphic overlays, the person perceives the alignmentoverlay 530 c to be at a different height (such as higher) than the sideoverlays 530 a, 530 b (which appear to extend along the ground or roadsurface rearward of the vehicle) and generally at or near the perceivedheight of the upper side overlays 531 a, 531 b to convey threedimensional information to the person viewing the display. Althoughseveral optional configurations and combinations of side overlays andcenter hitch alignment overlays are shown in FIGS. 4-14, clearly othercombinations and configurations may be implemented while remainingwithin the spirit and scope of the present invention.

The side overlays and the alignment overlay may be static overlays ormay be adjusted or curved or reconfigured (such as in response to asteering wheel angle of the vehicle or the like) to provide a dynamicoverlay when the vehicle is being driven in reverse toward the traileror targeted object. The side overlays and the centrally locatedalignment overlay may be activated or generated or displayed in responseto a user input, such as a button or other control input that isactivated by the driver of the vehicle when it is desired to back up thevehicle toward a targeted object or trailer hitch. Optionally, forexample, the displayed image may be activated in response to the vehiclebeing shifted into reverse and may include side overlays 30 a, 30 b(discussed above), and may provide the three dimensional side overlaysand the centrally located hitch alignment overlay in response to theuser input. The side overlays and/or the centrally located hitchalignment overlays may be displayed in different colors or differentintensities or may be intermittently displayed or the like based on adistance between the vehicle and the trailer hitch or targeted object,such as discussed above with respect to side overlays 30 a, 30 b, whileremaining within the spirit and scope of the present invention. Theoverlays (including the side overlays and the alignment overlay oroverlays) are arranged or displayed so that, when a person is viewingthe displayed image and graphic overlays, the overlays convey threedimensional information to the person viewing the display (such as byproviding an alignment overlay at a different perceived height than atleast a portion of the side overlays or by providing an alignmentoverlay and/or side overlays that are perceived as three dimensionalobjects, such as solid or transparent or partially transparent objects,or by providing any other suitable arrangement of overlays to conveythree dimensional information or depth to the person viewing thedisplayed images).

Thus, the imaging system and graphic overlays of the present inventionprovide enhanced cognitive awareness by the driver of any objectsdetected rearward of the vehicle when the vehicle is shifted into areverse gear and as the driver drives the vehicle in a reversedirection. The change in color or change in intensity or intermittentactuation of the graphic overlays on a segment-by-segment basis providesboth a spatial cue to the driver of the presence of and proximity of anobject rearward of the vehicle and a spectral cue to the driver of thepresence of and proximity of an object rearward of the vehicle, so thatthe driver is readily aware of an object that is detected rearward ofthe vehicle and is readily aware of the proximity of the detected objectto the rear of the vehicle. The centrally located hitch alignmentoverlay further enhances the discernability of the location of trailerhitch or targeted object to further assist the driver in maneuvering thevehicle in a rearward direction toward the trailer hitch or targetedobject.

Optionally, and desirably, the perspective and optical distortion may beaccounted for in the generated graphic overlays to provide more accuratespatial cues to the driver. For example, the orientation and arrangementand shape of the overlays as displayed on the display device or videoscreen may be calculated or displayed based on the distortion of thelens and the angle of the camera on the vehicle. For example, theoverlays may be curved as displayed to generally match the curvature ordistortion of the image as captured by the camera and wide angle lens atthe rear of the vehicle, and/or curved to generally match or account forthe perspective of how the camera is mounted on the vehicle. Thus, thearrangement or shape of the graphic overlays in the displayed image istailored specifically to the particular lens (such as a wide angle lens)used on the camera and to the vehicle mount (such as height and angle)of the camera at the rear of the vehicle.

Optionally, the graphic overlays may be adjustably displayed dependingon objects or other vehicles detected rearward of the vehicle. Forexample, if an object is detected rearward of the vehicle, the graphicoverlays may extend rearward from the subject vehicle toward thedetected object in the displayed image and may be ‘cropped’ so that thegraphic overlays are not shown at the detected object. Thus, the vehiclepath (static) overlays and/or vehicle path trajectory (dynamic-curve oradjust in response to turning of vehicle such as shown in FIG. 3)overlays may appear to “disappear” under the detected object or vehicle.Optionally, the graphic overlays may be adjusted in response to anobject detection system, such as an ultrasonic sensing system or theimaging system with image processing of captured images to detectobjects in the captured images or other object detecting devices orsystems, whereby, in response to a detection of an object or vehicle inthe path of the subject vehicle, such that the vehicle path may nolonger be valid or safe, the graphic overlays may be truncated orcropped or shortened so as to end at a location generally at thedetected object in the displayed image.

For example, if the system is coupled with an object detection system,such as ultrasonic sensors or other ranging device or the like, and ifan object (such as a bumper of another vehicle) is detected at adistance rearward of the subject vehicle (such as about 1 meter rearwardof the subject vehicle), the graphic overlays may be cropped so thatthey “disappear” under the detected object or vehicle (or, in otherwords, the graphic overlays are shown in the displayed image from therear of the subject vehicle toward the detected object or vehicle, andare not shown in the displayed image from the detected distance of thedetected object or vehicle and rearward from that detected distance orlocation). Such a graphic overlay adjustment provides the driver of thesubject vehicle, when viewing the displayed image with graphic overlaysestablished thereon, an impression that the overlays have gone under thedetected object or vehicle that is detected by the ranging device so asto enhance the cognitive awareness of the objects rearward of thesubject vehicle during a rearward maneuvering process.

Optionally, and preferably, the image sensor includes or utilizes a wideangle lens that focuses a wide angle rearward field of view (such as awide angle horizontal field of view of preferably greater than 120degrees wide and more preferably greater than 135 degrees wide and morepreferably at least about 145 degrees wide) onto an image plane of theimage sensor. The wide angle lens preferably comprises a multi-elementlens having multiple lens optics or elements, such as seven opticsarranged along an optic path to focus the rearward wide angle field ofview at the imaging plane while reducing distortion of the focusedimage. Optionally, for example, the wide angle lens may include sevenoptic elements, with some of the elements being glass elements (such asan outer wide angle element being a glass element) and some of theelements being plastic elements. Optionally, for example, the wide anglelens may include five optic elements, with three of the elements beingglass elements (including the outer most element being curved glass andthe inner most element being a flat glass coated with infrared cutofffilter and anti-reflection filter), and with two of the elements beingplastic elements. Preferably, at least one of the optic elementscomprises a plastic aspheric element, and more preferably, two of theoptic elements comprise plastic aspheric elements and five of the opticelements comprise glass spherical elements. Such a lens assembly mayutilize aspects of the lenses described in U.S. Pat. Nos. 6,922,292;6,757,109; 6,590,719; and 6,201,642, which are hereby incorporatedherein by reference in their entireties.

The lens of the present invention thus may have at least one plasticelement, which may provide a cost savings and a reduction in overalllens length. The refractive or diffractive lens element or elements maybe made by injection molding or other low cost means. The plastic lenselements may comprise a polycarbonate material or acrylic material orany other suitable plastic or polymeric material. The lens includesnon-spherical, refractive or diffractive optic elements, and may have astop or aperture within the lens to correct distortion in the focusedimage.

For example, a lens assembly may be provided that is approximately 25 to26 mm in overall length (from the image plane to the outer end of thelens assembly. The lens elements may be provided in a housing or barrel(such as an aluminum barrel) that may be threaded onto the circuit boardthat has the image sensor established thereon or threaded onto a housingor mounting portion of an image sensor module. The lens may have a capdiameter of about 12 mm to about 23 mm, with about a 1.0 to 2.5 mmradius on the outer most edge of the lens cap. The lens may provide awide angle field of view of about 130 degrees in the horizontal andabout 105 degrees vertical when the image sensor and lens are mounted atthe vehicle, and may have an active sensor area having about a 3.584 mmhorizontal dimension and about a 2.688 mm vertical dimension and about a4.480 diagonal dimension.

The image sensor may comprise a pixelated image sensor (such as a CMOSsensor or the like) having a 640×480 array (or other size arraydepending on the particular application) of pixels that are about 5.6μm×5.6 μm. The lens may provide an “F-stop” of about 2.0 (or lower)+/−5percent, and may have a relative illumination at greater than 50 percentat full field. Preferably, the lens provides a reduced or loweredoptical distortion (geometrical), preferably better than about −45percent. The lens may have a modulation transfer function (a measurementof lens resolution quality) of greater than about 0.6 at 45 lp/mm onlens axis (zero degrees) and greater than about 0.25 at 45 lp/mmoff-axis (between zero degrees and 60 degrees).

The lens preferably includes an integrated infrared (IR) cutoff coatingor filter that provides about 85 percent or greater transmission oflight therethrough over the visible spectrum (about 400 nm to about 700nm) of light, and about 50 percent or lower transmission of lighttherethrough for light having wavelengths greater than about 700 nm andless than about 400 nm. The IR cutoff coating may be applied to onesurface of one of the optical elements of the lens. Optionally, the IRcutoff filter may be also applied to a separate glass element, which maybe flat and thin and may be located as the last element of the lensassembly. The lens may provide a wide angle field of view and may beable to focus on an object within about 30 cm or thereabouts from theouter end of the lens. The lens preferably has an anti-reflective (AR)coating on at least some of the surfaces of the optic elements andpreferably on all of the surfaces of the optic elements. The lens mayhave a variation of effective focal length of less than +/−5 percent.The lens preferably has an image circle diameter of greater than about4.8 mm and the angle between the lens' optical axis and a barrelreference diameter axis is preferably less than about 1.5 degrees.

The lens is robust and capable of withstanding the extreme climate andenvironmental conditions that it may encounter at the exterior portionof a vehicle. For example, the lens may be operational at up to about 85degrees Celsius, and may be able to withstand exposure up to 125 degreesCelsius, The lens is capable of withstanding high temperature and highhumidity endurance tests and can withstand exposure at 95 degree Celsiusand 85 percent relative humidity for 1200 hours or more. The lens isalso capable of withstanding mechanical shock tests (such as 6 shockpulses with 100 g and 10 ms half-sine pulses, one in each oppositedirection of three perpendicular axes) and vibration tests (such as avibration test of an RMS acceleration value of about 27.8 m/s² for about27 hours in the x-axis, about 27 hours in the y-axis and about 81 hoursin the z-axis. Because the outer optic element is exposed to harshconditions at the rear of the vehicle, the lens comprises a sealed lensand is capable of passing the required OEM testing, such as a car washspray test and a heavy splash shower test and a mist test and a saltmist test and a dust test and the like. The lens can also withstand adunk or soak test where the lens is soaked in 95 degrees Celsius waterfor at least 1 hour and then the upper body of the lens is dunked orsoaked in icy water at a depth of 15 cm for at least 30 minutes. Thelens is also capable of withstanding exposure to various chemicals. Forexample, the outer glass surface of the lens and the outer exterior bodyof the lens may be resistant to automatic transmission fluid, hypoidlubricant, hydraulic fluid, power steering fluid, differentiallubricant, central hydraulic fluid, engine oil, engine wax protective,engine coolant/Ethylene Glycol, gasoline, diesel fuel, kerosene,bio-diesel/Methanol based fuel, brake fluid, windshield washer fluid,window glass cleaner, car wash cleaner/soap solution, car wax andsilicone protectants, leather wax, battery acid-dilute sulfuric acid,and calcium chloride, and/or other chemicals the lens may encounter atthe vehicle assembly plant and/or during its life on the vehicle.

Because the outer glass optic of the lens assembly is positioned towardan exterior of the vehicle such that an outer surface of the outer glassoptic may be exposed to the elements outside of the vehicle, the outersurface may be coated with a protective coating to protect the outeroptic from deterioration or damage due to exposure to the elements. Sucha protective coating may comprise an organic haricot, an inorganicharicot, or an organic/inorganic compound or the like. Additionally, theprotective coating may include ultraviolet absorbers or stabilizers toprotect the outer optic from UV radiation degradation. Such a coating orother additional coating or coatings substantially reduces thesusceptibility of the outer optic to abrasion and/or ultravioletdegradation, thereby providing a substantially clear and durableglass-like appearance to the outer optic. Optionally, the hard coatingor protective coating may be of the type described in EPC ApplicationNo. 98650039.5 for VEHICULAR COMPONENT ASSEMBLY WITH HARD COATEDELEMENT, published on Jan. 20, 1999 under Publication No. EP 0892209,the disclosure of which is hereby incorporated herein by reference inits entirety.

Optionally, the outer optic may be heatable to defog or defrost itssurfaces. For example, the outer optic may be formed by a glass materialand coated with a transparent conductive coating (such as an indium tinoxide (ITO) transparent conductor or a doped tin oxide or the like), ormay include a wire mesh or a conductive coating mesh or the like,whereby heat is generated by energizing the conductive coating or meshvia a power source. Optionally, the outer surface of the outer optic maybe coated with an anti-soiling or anti-wetting coating, such as asilicone material, which may provide either a hydrophobic or hydrophilicproperty to the exposed outer surface of the outer optic.

Optionally, the lens body (barrel and optic elements) may be heatedelectrically to limit or substantially preclude moisture from beingtrapped inside the lens from condensation. When the lens is assembled inan environment that humidity/moisture level is not tightly controlledbelow a certain level, the air packet between optic elements inside thelens typically has water molecules in the form of moisture. When thelens is exposed to a temperature that is lower than the dew pointtemperature of the moisture density level, moisture condensation occurswithin the lens, and a small cluster of water droplets may form on thesurfaces of the optic elements inside the lens. The condensation orwater droplets may blocks or attenuate light passing through the lensand thus may degrade the lens and furthermore may reduce or impair thecamera performance. In some situations, the condensation water may droponto the camera circuitry or circuit board or PCB and may cause anelectrical malfunction or electrical shortage. Optionally, a means formeasuring the temperature of the lens, such as a temperature sensor,such as a thermal couple or the like, may be used to determine thetemperature at or within the lens, while a control means may beresponsive to the temperature signal and may activate a heating currentor voltage at or in the lens to heat the lens or lens elements above athreshold level in response to a detection or determination that thetemperature at or in or of the lens is dropping below the dew pointlevel. Thus, the lens or lens elements may be heated to remain above thedew point temperature so as to limit or substantially avoid moisturecondensation from occurring within the lens.

Optionally, a lens body may be filled with a gas that is without watermoisture. For example, a dry air, nitrogen gas or helium gas (or othersuitable non-moisture gas or air) can be filled inside the lens body.The lens is then sealed to keep the moisture from getting inside lensfrom outside. The lens can be assembled in an enclosed environment thatis filled with the gas without water moisture, such as a dry air,nitrogen or helium gas or the like. Optionally, another means foraddressing the moisture issue may provide breathing paths within thelens. For example, the lens body can be designed to have one or morebreathing paths by which air and water molecules can travel from all ofthe inside cavities of the lens to the outside of the lens, such asthrough the lens bottom or other body wall or housing wall. Optionally,by providing some moisture absorbing material inside the camera cavity,the water molecules that may originally exist inside the lens body canbe absorbed by the moisture absorbing material.

Optionally, the lens of the present invention may have a lens barrel anda lens cap made of plastic material or one made of a plastic materialand one made of a metal material. Optionally, the plastic lens barreland lens cap can be made in injection molding process, and thus mayprovide a lower cost of the lens. Optionally, the plastic parts can bemade via injection molding, and thus can be made at a lower cost and ina higher volume.

An automotive camera is typically exposed to a wide temperature range(for example, from about −40 degrees C. to about 85 degrees C.) and istypically required by the automobile manufacturer to maintain sufficientor good lens focus throughout the expected temperature ranges to whichthe vehicle may be exposed. Because using an injection molded plasticmaterial may reduce the overall cost of the camera, plastic materialsmay be selected for use in the lens barrel and the camera body,including the lens holder. Optionally, a metallic material, such asaluminum and/or copper and/or the like, may be selected for use in thelens barrel or camera body. The material selection for the lens and/orcamera mechanical design may be selected to achieve good or sufficientlens focus over the entire temperature range to which the camera may besubjected.

The use of a plastic material in the camera body and lens barrel maypose a challenge in maintaining the focus of the camera because of thelarger or enhanced thermal expansion (as compared to metallic material),which can potentially cause the lens to become less focused. In order toreduce the temperature effects on the lens focus, the type of material(plastic or metal) may be selected to match the camera lens holdermaterial type (plastic or metal). The lens is mounted to the camera lensholder at the top or outer part of the lens, or at the part that iscloser to the lens outer surface. When the camera is subjected to asubstantial change in temperature, the expansion or contraction of theplastic lens barrel or camera lens holder may move the lens focal planeaway from the imager sensing plane, and thus may cause de-focusing orblurriness in the captured images. To counter this potential problem,the material for the lens barrel and/or the camera lens holder may beselected to have the same or similar thermal expansion. For example, asame or similar plastic material having the same or similar thermalexpansion performance characteristics or properties may be selected forboth components.

Thus, when the camera is exposed to temperature changes, such as to anextreme hot (or cold) temperature, the expansion (or contraction) of thelens barrel happens generally simultaneously with the expansion (orcontraction) of the camera lens holder, but the expansions (orcontractions) expand/contract in the opposite directions of one another.The net result is a cancellation or reduction or offsetting of thethermal expansion of the components and, thus, enhanced focusing for thecombined system over a broad range of temperatures. In other words, theresult may be an elimination or reduction or offset of the movement oflens focal plane. Optionally, a metal lens barrel with metal camera lensholder may be selected to provide a similar benefit of maintaining focusunder a wide temperature range, because of their similar thermalexpansion characteristics.

The above design and material selection thus may maintain good orsufficient focus over a wide temperature range, which is very importantto automotive camera applications. However, when combining a metal lensbarrel with the plastic camera lens holder, or vise versa, because ofthe substantial differences in thermal expansion characteristics betweenthe plastic and metal materials, there may be a reduced or nocancellation or reduction or offset of the movement of lens focal plane.The camera thus may become out of focus or less focused at extremetemperature conditions. Thus, the matching of the lens barrel materialand the camera lens holder material, such as matching plastic withplastic or metal with metal, is a desirable selection for enhancedperformance of the camera throughout a wide temperature range.

The wide angle lens includes corrective refractive and diffractiveelements positioned along the optic path between the wide angle outeroptic and the image sensor. Additional corrective elements may also beincluded to further correct color or distortion within the refractedimage. Diffractive elements are preferably included to correct the colorfocusing within the refracted image. The diffractive element or elementsmay further include an aspheric refractive optic element attachedthereto or on an opposite surface of a respective optic element tofurther correct for distortions and aberrations within the focusedimage. Optionally, these refractive optic elements may be transparentplastic optic elements, having a varying width and surface curvature,such that the image passing through the refractive lens is refracted invarying degrees, corresponding to the magnitude of distortion in theimage received by the refractive optics. Therefore, either bythemselves, or in conjunction with the image sensor and/or imageprocessing of the captured images, the refractive and/or diffractiveoptic elements of the lens function as a means for correctingdistortions within the image.

The image system may further reduce image distortion via otherdistortion reducing means, such as by utilizing non-uniformlydistributed pixels at the image array sensor (as discussed below), orsuch as via an on-imager processing unit (such as a system on chip orthe like), such as an FPGA, CPLD, DSP, microprocessor or the like, andassociated software, or such as via display means where the distortioncorrection is done in the displayed image via a non-uniformlydistributed pixelated array of the display or via image processinghardware or software of the display device or module.

It is further envisioned that the wide angle imaging system may includeelectronic image processing or distortion correction to further correctthe image displayed on the display. The distortion correction processmay be performed by an image processing algorithm or by pixeldistribution variation on image capture device and/or an associateddisplay. Preferably, the system may display a vehicle reference, such asa rear bumper of the vehicle, as is seen outside the vehicle, so as toprovide a sense of correctness to the image displayed to the driver. Forexample, an image processing algorithm may be provided which ensuresthat straight horizontal and/or vertical lines in the scenic informationexteriorly of the vehicle are displayed as straight horizontal and/orvertical lines in the image displayed on the display. Such processingallows the imaging system to implement the wide angle lens and stillprovide a substantially realistic display image to the driver.

Optionally, the image sensor and/or image processing of the capturedimages may be capable of reducing distortion of the captured image. Suchdistortion reduction may be accomplished by a non-uniform array ofphotosensing pixels of the image sensor, such as described in U.S. Pat.No. 5,796,094, which is hereby incorporated herein by reference in itsentirety. The non-uniform array may have a coarse distribution of pixelsin a center region, with a finer distribution of pixels along theperimeter regions of the captured image. Such non-uniform pixelation ofthe imaging array receives the focused image and varies correspondinglywith the magnitude of distortion in the image refracted through thelens.

Optionally, the image system of the present invention may providecorrection of image perspective distortion. Image distortion from thecamera with a wide angle lens includes not only lens geometricaldistortion (such as pincushion and barrel distortions), but alsoperspective distortion, which is caused not by lens distortion, butrather is a perspective distortion or view change whereby an objectappears in different sizes when it is located at different distancesfrom the camera. This type of distortion may also be corrected via oneor more refractive and/or diffractive optic elements of the lens so asto function as lens-based means for correcting distortion. Optionally,the perspective distortion may also or otherwise be corrected via animager-based means, image processing means and/or display means forcorrecting distortion, such as those described above.

Optionally, the graphic overlays may be superimposed or projected ontothe displayed image based on the image distortion. For example, thesystem may change or adjust the graphic overlay projection (such as theangle, curvature, thickness, length and/or other characteristics of thelines or overlays) of an overlay pattern in accordance with the positionof the overlay pixel at the displayed image. The graphic overlayprojection can be designed based on the lens distortion curve. When animage processing distortion correction is applied to the camera, thecamera system level distortion curve can be known by design ormeasurement, and the graphic overlay projection can be designed based onthe camera system level distortion curve.

Thus, the wide angle rearward image system of the present inventionprovides a wide angle rearward field of view with reduced distortion tothe driver of the vehicle so that the images displayed to the driver arereadily recognized by the driver when the driver is reversing thevehicle. The image system preferably provides graphic overlays tofurther enhance the driver's cognitive awareness of detected objectsrearward of the vehicle. Although described as a rearward facing imagingsystem, aspects of the present invention are equally suited for otherimaging systems of vehicles, such as side object detection systems orblind spot detection systems or forward facing imaging systems, such aslane departure warning systems and/or the like.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

The invention claimed is:
 1. An imaging system for a vehicle, saidimaging system comprising: an imaging sensor, wherein said imagingsensor comprises a two-dimensional array of light sensing photosensorelements, and wherein said imaging sensor has a rearward field of viewwhen mounted at the vehicle; a display device for displaying imagesrepresentative of image data captured by said imaging sensor; andwherein said imaging system generates a plurality of graphic overlays onsaid displayed images to enhance the driver's cognitive awareness of anobject rearward of the vehicle, said graphic overlays comprising aplurality of graphic overlay segments, said graphic overlay segmentsconveying three dimensional information to a person viewing saiddisplayed images and said graphic overlays.
 2. The imaging system ofclaim 1, wherein said imaging system adjusts at least one of a color, anintensity and a rate of flashing of at least one of said plurality ofgraphic overlay segments in response to an object being detectedrearward of the vehicle and within a distance threshold.
 3. The imagingsystem of claim 1, wherein said each of said overlays comprises at leasttwo segments, and wherein said imaging system is operable to adjust atleast one of a color, an intensity and a rate of flashing of a firstgraphic overlay segment in response to an object being detected rearwardof the vehicle and within a first distance threshold, and wherein saidimaging system is operable to adjust at least one of a color, anintensity and a rate of flashing of a second graphic overlay segment inresponse to an object being detected rearward of the vehicle and withina second distance threshold.
 4. The imaging system of claim 1, whereinsaid graphic overlays are displayed so as to generally correspond to thedistortion in said displayed images.
 5. The imaging system of claim 1,wherein said graphic overlays are adjusted in response to an objectdetection system so as to not be displayed at areas of said displayedimage that are representative of a detected object.
 6. The imagingsystem of claim 1, wherein said imaging system generates an alignmentgraphic overlay on said displayed images to assist the driver of thevehicle to align the vehicle with a targeted object.
 7. The imagingsystem of claim 6, wherein said alignment graphic overlay comprises acentrally located overlay extending rearward from a centerline of thevehicle.
 8. The imaging system of claim 7, wherein said imaging sensoris offset from the centerline of the vehicle, said alignment graphicoverlay being displayed as a sideward perspective view of a virtualalignment graphic overlay to enhance the driver's cognitive awareness ofthe alignment graphic overlay.
 9. The imaging system of claim 7, whereinsaid graphic overlay segments comprise a side graphic overlay segmentextending upward and inward along each side of the image so as to beindicative of segments extending rearward from the sides of the vehicle,and wherein said alignment graphic overlay is viewed by the driver ashaving a different virtual height dimension than said side graphicoverlay segments to enhance the driver's cognitive awareness of saidgraphic overlays.
 10. The imaging system of claim 1, wherein saidgraphic overlay segments comprise a graphic overlay segment extendingupward and inward along each side of the image so as to be indicative ofsegments extending rearward from the sides of the vehicle, and whereinsaid graphic overlay segments are viewed by the driver as having avirtual height dimension to enhance the driver's cognitive awareness ofsaid graphic overlays.
 11. The imaging system of claim 10, wherein saidgraphic overlay segments are displayed as three dimensional graphicoverlay segments having a virtual height and a virtual thickness. 12.The imaging system of claim 11, wherein said imaging system generates analignment graphic overlay on said displayed images to assist the driverof the vehicle to align the vehicle with a targeted object, and whereinsaid alignment graphic overlay comprises a centrally located overlayextending rearward from generally a centerline of the vehicle.
 13. Theimaging system of claim 11, wherein said three dimensional graphicoverlay segments comprise first and second segments, said first segmentcomprising a different color segment than said second segment.
 14. Theimaging system of claim 13, wherein a color scheme of said first andsecond segments is selectively adjustable by the driver of the vehicleto configure said graphic overlays for the vision characteristics of thedriver of the vehicle.
 15. The imaging system of claim 11, wherein saidthree dimensional graphic overlay segments are displayed such thatimages of the rearward scene rearward of and toward either side of thevehicle are viewable through said three dimensional graphic overlaysegments.
 16. An imaging system for a vehicle, said imaging systemcomprising: an imaging sensor, wherein said imaging sensor comprises atwo-dimensional array of light sensing photosensor elements, and whereinsaid imaging sensor has a rearward field of view when mounted at thevehicle; a display device for displaying images representative of imagedata captured by said imaging sensor; wherein said imaging systemgenerates a plurality of graphic overlays on said displayed images toenhance the driver's cognitive awareness of an object rearward of thevehicle said graphic overlays comprising a plurality of graphic overlaysegments, said graphic overlay segments conveying three dimensionalinformation to a person viewing said displayed images and said graphicoverlays; and wherein said imaging sensor has a lens for focusing afield of view onto an image plane of said imaging sensor, and whereinsaid lens comprises a multi-element wide angle lens, and wherein saidmulti-element wide angle lens comprises at least one plastic asphericelement and wherein an outer element comprises a wide angle glasselement, and wherein one of (a) said multi-element wide angle lenscomprises a seven element lens, said seven element lens comprising twoplastic elements and five glass elements, and (b) said multi-elementwide angle lens comprises a five element lens, said five element lenscomprising two plastic elements and three glass elements.
 17. Theimaging system of claim 1, wherein said imaging sensor has a lens forfocusing a field of view onto an image plane of said imaging sensor, andwherein a material of a lens barrel and a material of a lens holder areselected to have substantially similar thermal expansion characteristicsto substantially offset thermal expansion of said lens barrel and saidlens holder to generally maintain the focus of said imaging sensor. 18.An imaging system for a vehicle, said imaging system comprising: animaging sensor, wherein said imaging sensor comprises a two-dimensionalarray of light sensing photosensor elements, and wherein said imagingsensor has a rearward field of view when mounted at the vehicle; adisplay device for displaying images representative of image datacaptured by said imaging sensor; wherein said imaging system generates aplurality of graphic overlays on said displayed images to enhance thedriver's cognitive awareness of an object rearward of the vehicle, saidgraphic overlays comprising a plurality of graphic overlay segments;wherein said graphic overlay segments comprise a side graphic overlaysegment extending upward and inward along each side of the image so asto be indicative of segments extending rearward from the sides of thevehicle and wherein said graphic overlay segments comprise an alignmentgraphic overlay segment to assist the driver of the vehicle to align thevehicle with a targeted object; and wherein at least one of said graphicoverlay segments is viewed by the driver as having a virtual heightdimension to enhance the driver's cognitive awareness of said graphicoverlays, and wherein said at least one of said graphic overlay segmentsis displayed as a three dimensional graphic overlay segment having avirtual height and a virtual thickness.
 19. The imaging system of claim18, wherein at least one of said graphic overlay segments comprises atleast two segment portions, and wherein said imaging system is operableto adjust at least one of a color, an intensity and a rate of flashingof a first segment portion in response to an object being detectedrearward of the vehicle and within a first distance threshold, andwherein said imaging system is operable to adjust at least one of acolor, an intensity and a rate of flashing of a second segment portionin response to an object being detected rearward of the vehicle andwithin a second distance threshold.
 20. The imaging system of claim 18,wherein said alignment graphic overlay comprises a centrally locatedoverlay extending rearward from a centerline of the vehicle.
 21. Theimaging system of claim 20, wherein said alignment graphic overlay isdisplayed as a three dimensional graphic overlay segment, and whereinsaid imaging sensor is offset from the centerline of the vehicle, saidalignment graphic overlay being displayed as a sideward perspective viewof a virtual alignment graphic overlay to enhance the driver's cognitiveawareness of said alignment graphic overlay.
 22. The imaging system ofclaim 18, wherein said graphic overlays are adjusted in response to anobject detection system so as to not be displayed at areas of saiddisplayed image that are representative of a detected object.
 23. Theimaging system of claim 18, wherein said side graphic overlay segmentsinclude laterally extending horizontal linear marks to assist the driverin judging distances rearward of the vehicle.
 24. The imaging system ofclaim 18, wherein at least one of said graphic overlay segmentscomprises first and second segment portions, said first segment portioncomprising a different color segment than said second segment portion.25. The imaging system of claim 18, wherein said at least one threedimensional graphic overlay segment is displayed such that images of therearward scene rearward of and toward either side of the vehicle areviewable through said at least one three dimensional graphic overlaysegment.
 26. The imaging system of claim 25, wherein said imaging systemadjusts at least one of a color, an intensity and a rate of flashing ofat least one of said at least one three dimensional graphic overlaysegment in response to an object being detected rearward of the vehicleand within a distance threshold.
 27. The imaging system of claim 18,wherein a color scheme of said graphic overlay segments is selectivelyadjustable by the driver of the vehicle to configure said graphicoverlay segments for the vision characteristics of the driver of thevehicle.